
1 

MAXIMUM LIKELIHOOD ESTIMATION 
OF JPEG QUANTIZATION VALUES 

BACKGROUND OF THE INVENTION 

1. Field of Invention 

5 This invention generally relates to processing decompressed digital images 

compressed using a quantizing compression technique. 

2. Description of Related Art 

Data compression is required in data handling processes, where too much data 
is present for practical applications using the data. Commonly, compression is used in 

10 communication links to reduce the transmission time or required bandwidth. 

Similarly, compression is preferred in image storage systems, including digital 
printers and copiers, where "pages" of a document to be printed are stored temporarily 
in precollation memory. The amount of media space on which the image data is 
stored can be substantially reduced with compression. Generally speaking, scanned 

15 images, i.e., electronic representations of hard copy documents, are often large, and 
thus make desirable candidates for compression. 

The image compression standard disseminated by the Joint Photographic 
Experts Group (JPEG) committee is a compression technique which reduces data 
redundancies based on pixel-to-pixel correlations. Generally, a photographic image 

20 does not change very much on a pixel-to-pixel basis and therefore has what is known 
as "natural spatial correlation." In natural scenes, correlation is generalized, but not 
exact. Noise makes each pixel somewhat different from its neighbors. 

SUMMARY OF THE INVENTION 
The methods and apparatus of this invention seek to enhance conventional 

25 methods for processing decompressed images. In a conventional method for 

processing images, an image source provides the compressed image data to the 
system. The image source can be an input device such as a camera or scanner, a 
transmission channel or a storage device. The compressed image data is input to a 
decompression unit that decompresses the image data to reconstruct the image. The 

30 decompressed image is sent to an image processing system that processes the 
decompressed image. 
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In processing decompressed images, such as decompressed JPEG-compressed 
images, for example, a quantization table is often required. However, the quantization 
information is often not available to the image processing system, especially when the 
image is decompressed remotely. In particular, if the image was decompressed apart 
5 from the image processing, the image processing system might not have available 
information indicating that the image has been JPEG compressed, for example, or 
information identifying or defining the quantization table(s) used to compress the 
image. 

This invention provides systems and methods that process images without 
10 requiring information on whether the image was previously compressed or how the 
image was compressed. 

This invention provides systems and methods that determine quantization 
values of one or more image blocks of a decompressed image. 

This invention separately provides systems and methods that are able to 
15 identify a compression method used to compress a decompressed image to be 
processed. 

The systems and methods of this invention are particularly useful to determine 
if a decompressed image was previously compressed at least in part using a JPEG 
compression technique. 
20 This invention separately provides systems and methods useable to process a 

JPEG compressed digital image. 

These and other features and advantages of this invention are described in or 
are apparent from the following detailed description of the various exemplary 
embodiments. 

25 BRIEF DESCRIPTION OF THE DRAWINGS 

The preferred embodiments of this invention will be described in detail, with 
reference to the following figures, wherein: 

Fig. 1 is a generalized functional block diagram of one exemplary embodiment 
of a system including an image processing system according to this invention; 
30 Fig. 2 shows a generalized functional block diagram of one exemplary 

embodiment of the image output device of Fig. 1 according to this invention; 
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Fig. 3 shows a generalized functional block diagram of one exemplary 
embodiment of the quantization table estimator of Fig. 2 according to this invention; 

Fig. 4 is a flowchart outlining an image processing method in accordance with 
this invention; 

5 Fig. 5 is a flowchart outlining in greater detail one exemplary embodiment for 

determining a quantization table of Fig. 4; and 

Fig. 6 is a flowchart outlining in greater detail one exemplary embodiment for 
determining a maximum likelihood estimation of Fig. 5; and 

Fig. 7 is a flowchart outlining in greater detail one exemplary embodiment for 
10 determining the likelihood of Fig. 6. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
The systems and methods of this invention identify a compression method of a 
decompressed image. In processing decompressed images compressed using a 
quantizing compression technique, because often only the decompressed image data is 
15 input to the image processing system, the quantization table and/or other data related 
to the compressed data are not available to the image processing system. 

In one exemplary embodiment of the systems and methods of this invention, 
the decompressed image is analyzed to determine whether the decompressed image 
was JPEG compressed. A quantization table, as further described below, is obtained 
20 during image processing. Upon image processing, the quantization table is 

regenerated by determining the maximum likelihood of the quantization table values. 
To determine the regenerated quantization table, the DCT coefficients of the re- 
compressed image are determined, using a processor to process the decompressed 
image data using JPEG compression techniques. Thus, image processing can be 
25 performed even if the quantization information might not be available or the image 
was decompressed remotely from the compression system. 

The Discrete Cosine Transform (DCT) is central to a JPEG compression 
process performed on the image data. For example, when a Discrete Cosine 
Transform is performed on values for a block of image data, such as an 8x8 pixel 
30 block of image data, for example, the result is a set of 64 DCT coefficients. The DCT 
coefficients represent the amplitudes of 64 respective orthogonal waveform 
components that together define the values for all 64 pixels in the 8x8 pixel block. An 
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inverse Discrete Cosine Transform performed on the 64 coefficients will reproduce 
the original 8x8 pixel block values. 

The advantage of using these 64 coefficients instead of the 64 original image 
values is that each coefficient provides the magnitude of an orthogonal waveform 
5 representing a different spatial frequency. Smooth textured blocks have low pixel-to- 
pixel variation. Thus, many zero-value "high-frequency" DCT coefficients are likely. 
For example, performing a Discrete Cosine Transform on a block of 64 pixels having 
identical values will result in one nonzero coefficient and 63 zero value coefficients. 
Further, if the coefficients are ordered by spatial frequency, long strings of zero-value 

10 coefficients will result. 

One advantage of using data with long zero-value strings is obtaining greater 
data compression, for example, when using run-length encoding and/or entropy 
encoding such as Huffman-type encoding. For this reason, when a Discrete Cosine 
Transform is determined for a pixel block, it is desirable to represent the coefficient 

15 for high spatial frequencies with less precision. This representation is done by a 
process called quantization. Quantization is basically a process for reducing the 
precision of the DCT coefficients. Precision reduction is extremely important, since 
lower precision almost always implies a lower bit rate in compression. 

A DCT coefficient is quantized by being divided by a nonzero positive integer 

20 called a quantization value, and truncating or rounding the quotient, or the quantized 
DCT coefficient, to the nearest lower integer. In order to reconstruct or dequantize 
the DCT coefficient, the quantized DCT coefficient is multiplied by the quantization 
value. Since some precision is lost in quantizing, the reconstructed DCT coefficients 
are approximations of the values before quantization. 

25 The original DCT coefficients for image data blocks are not recoverable. They 

are best approximated by the dequantized values {Y(m,n)}. {Y(m,n)} are retrieved as 
multiples of the quantization values from the quantization table. In other words the 
dequantized DCT coefficients Y(m,n) can be expressed as k-q(m,n), where q(m,n) is 
the (m,n) entry of the quantization table, and k is an integer. In inverse Discrete 

30 Cosine Transform processing, the dequantized DCT coefficient {Y(m,n)} is 
transformed to produce the decompressed image block {y(ij)}. 
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As discussed above, inverse Discrete Cosine Transform processing is 
theoretically reversible with Discrete Cosine Transform. In other words, a Discrete 
Cosine Transform of {y(ij)} would produce exactly {Y(m,n)}. However, in practice, 
the Discrete Cosine Transform only generates {Y*(m,n)}, which is an approximated 
5 version of {Y(m,n)} . Common sources of errors may include, but are not limited to 
the pixel value of the decoded image block y(i j), which is typically an integer, being 
rounded from a real number; the decoded image block y(i j), which may be a number 
greater than 255 or smaller than 0, being truncated to 255 or 0, respectively; or 
transforms being determined with limited accuracy. In the methods and systems of 

1 0 this invention, at least the rounding error is determined and used in determining the 
maximum likelihood of the quantization values. 

Since blocks with uniform intensity and blocks with truncation are treated 
separately in estimation, those blocks that are analyzed using the systems and methods 
of this invention should be non-uniform and should not have been truncated. For those 

15 blocks, a likelihood function can be established as: 

L(q) = Z s log {p[Y s *(m,n); q]}, (1) 

where: 

s is the s-th block in the estimation. 
The probability function for each block is: 
20 p[Y*(m,n); q(m,n)] = p Y [r(m,n) q(m,n)] Z k p x [Y*(m,n) | k q(m,n)]. (2) 

where: 

p Y [.] is a Gaussian or Laplacian distribution for (m,n) = 0, or (m,n) ^0, 
respectively, 

p x [.] is a Gaussian distribution, and 
25 r(m,n) = round [Y*(m,n) / q(m,n)]. (3) 

The summation is over a range of all integers k such that: 

|Y*(m,n) - k q(m,n) | < D(m)D(n). * (4) 
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where D(.) is a function defined to be: 



D(x) = 2; forx=0or4, 
2 cos 7i/4; for x— 2 or 6, 

2 cos 7i/4 cos 7i/8; for x odd. (5) 

5 Accordingly, the maximum likelihood estimation q*(m,n) of the quantization 

value q(m,n) in the quantization table regenerated using the systems and methods of 
this invention is determined as: 



q*(m,n) - arg max q L(q) (6) 
= arg max q {Z s log {Z k p[Y s *(m,n) | k q]} 
10 -Nlog[a*(m,n)]}, 



where: 

N is the total number of blocks used in estimation; and 
a* (m,n) is the estimation of parameter for Gaussian distribution. 
Specifically, 

15 o* (m,n) - [S s r s 2 (m,n)/N] 1/2 , for (m,n) = (0,0); 

a* (m,n) = Z s |r s (m,n)| /N, for (m,n) ^ (0,0). (7) 

In one exemplary embodiment of the systems and methods of this invention, to 
reduce computation, each re-compressed DCT coefficient {Y*(m,n)} is rounded to an 
integer, which can be denoted as {Y f (m,n)}. The maximum likelihood estimation 
20 q*(m,n) can be shown as: 



q*(m,n) = arg max q {Z s log {S k p[Y' s (m,n) I k q]} - N log [a* (m,n)]» 

= arg max q {Z { N(i) log P d (i; q) + N log [a* (m,n)]} , (8) 

where: 

N is the total number of blocks used in estimation; and 
25 N(i) is the number of blocks, such that: 



Y' s (m,n) - q r s (m,n) | = i; and 



(9) 
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P d (i;q) is the probability conditioned satisfied by Eg. 9. 
In particular, P d (i; q) can be determined as 

P d (i;q) = ZjI(i+jq),and (10) 
I(i) = J (6/tc * exp[-6 x 2 ] dx, (1 1) 

5 where the integral limits are from i-0.5toi + 0.5. 

In one exemplary embodiment of the systems and methods of this invention, to 
further reduce the resources necessary to determine the quantization to table(s), not all 
the values are tested in estimating the maximum likelihood. In building a histogram 
for the rounded re-compressed DCT coefficient Y' (m,n), normally, the highest peak 

10 outside the main lobe (0 and its vicinity) corresponds to the quantization value q(m,n) 
or one of its multiples. According to this exemplary embodiment, the search is 
restricted to be the values Q, Q+l, and Q-l and their integer fractions, where Q is the 
highest peak outside the main lobe. Thus, according to this exemplary embodiment, 
the search is restricted to the highest peak outside the main lobe and the two adjacent 

15 levels. 

The image processing of a decompressed image can be performed by: 

1 ) For each image block, determine the maximum and the minimum values of 
the block. If the maximum value is 255 or the minimum value is 0, the block may 
contain truncated pixels. If the maximum value is equal to the minimum value, the 

20 block is uniform. Both kinds of blocks need to be excluded from further processing to 
avoid introducing unnecessary errors. 

2) A Discrete Cosine Transform is performed for each block not excluded in 
step 1). The results are rounded to integers as {Y f (m,n)}. 

3) For each DCT coefficient (m,n), steps 4-15 are performed. 

25 4) A histogram h is built from the rounded block value {|Y ? (m,n) |}. 

5) An index with the highest count in the histogram is determined for 
indices greater than 4. The highest count index is denoted as Q. This is the highest 
peak outside the main lobe. If the highest count is 0, the quantization value q(m,n) 
cannot be determined, and control returns to step 3. 

30 6) The maximum likelihood value is initiated as max_likelihood 
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= -oo 

7) For Ql = Q-l, Q, and Q+l, i.e. for the highest peak outside the main 
lobe and the two adjacent levels: 

8) For each value of x, to initially estimate q = Ql/x, 
5 where x = 1, 2, Ql: 

9) If q is an integer, then 

10) For z = q-l, q-2, ...,0: 

11) Calculate the number of blocks N(z) = Zj h(z + j q). 

12) If z> M and q-z >M and N(z) > 0, where M 
1 0 = min {q-l , round[D(m)D(n)] } , 

then return to step 8 for next q . 

13) Calculate likelihood = Z { N(i) log P d (i) + N log a*. 

14) If likelihood > max_likelihood, then set max_likelihood 
= likelihood and Est_q = q. 

15 15) The maximum likelihood estimation, MLE, of q(m 5 n) is given as 

Est_q. 

Fig. 1 shows a generalized functional block diagram of a system 100 that 
includes an image processing system 200 according to this invention. The system 100 
can include an image source 110 that may be any one of a number of different 
20 sources, such as a scanner, a digital copier or a facsimile device suitable for 

generating electronic image data, or a device suitable for storing and/or transmitting 
the electronic image data, such as a client or a server of a network. The electronic 
image data from the image source 1 10 is provided to a compressor 400 of the system 
100. 

25 In particular, in various exemplary embodiments, the compressor 400 

compresses the image using various compression operations associated with the JPEG 
compression standard to compress the image data within a block. However it should 
be appreciated any known or later developed compression technique that compresses 
and quantifies the image data on a block-by-block basis is equally relevant to the 

30 image processing systems and methods of this invention. In the compressor 400, the 
data may be operated on in any of a number of well-known bit- or byte-wise 
operations to accomplish the compression of the image data, wherein additional 
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information is used and/or generated (such as a dynamically adjusted quantization 
table) as the image is compressed. 

Once compressed, the compressed image data then is transferred to the 
channel or storage device 300. The channel or storage device 300 can be either or 
5 both of a channel device for transmitting the compressed image data to the decoder 
500 or a storage device for indefinitely storing the compressed image data until there 
arises a need to decompress the compressed image data. The channel device can be 
any known structure or apparatus for transmitting the compressed image data from a 
first apparatus implementing the compressor 400 according to this invention to a 
10 physically remote decompressor 500 according to this invention. Thus, the channel 
device can be a public switched telephone network, a local or wide area network, an 
intranet, the Internet, a wireless transmission channel, any other distributed network, 
or the like. 

Similarly, the storage device can be any known structure or apparatus for 
1 5 indefinitely storing compressed image data, such as a RAM, a hard drive and disk, a 
floppy drive and disk, an optical drive and disk, flash memory or the like. Moreover, 
the storage device can be physically remote from the encoder 400 and/or the decoder 
500, and reachable over the channel device described above. 

The compressed image data is then processed by the decompressor 500. The 
20 decompressor 500 receives the compressed image data from the channel or storage 
device 300 and recombines the blocks of decompressed image data in their 
corresponding position. Though the compressor 400 and the decompressor 500 are 
each shown in Fig. 1 as physically separate from the image processing system 200, it 
should be understood that the decompressor 500, and/or the compressor 400 may be 
25 different aspects, with the image processing system 200, of a single physical device. 

The decompressor 500 sends the reconstructed image to the image processing system 
200. 

Fig. 2 shows a generalized functional block diagram of one exemplary 
embodiment of the image processing system 200 according to this invention. The 
30 following description of the image processing system 200 assumes the image to be 
processed was compressed using a JPEG compression technique. It will be apparent 
to those of ordinary skill in the art, from the following description of the image 
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processing system 200, how to modify the image processing system 200 to deal with 
other compression techniques to perform the image processing according to this 
invention. Thus, because such modifications are readily apparent and predictable 
from the following discussion of the image processing system 200, additional 
5 descriptions of other compression techniques 1 implementations are not necessary and 
are thus omitted. 

The image processing system 200 includes an image blocking portion 220, a 
quantization table estimator 240 and an image processor 260. The image blocking 
portion 220 divides the decompressed image data into a plurality of MxM blocks or 

10 segments. The quantization table estimator 240 inputs the MxM blocks of image data 
and outputs one or more estimated quantization tables for the blocks to the image 
processor 260. The image processor processes the decompressed image data based on 
the one or more estimated quantization tables. 

Fig. 3 shows a generalized functional block diagram of one exemplary 

15 embodiment of the quantization table estimator 240. In the quantization table 

estimator 240, a Discrete Cosine Transform is performed on the MxM blocks of 
image data in a DCT transformer 242. A data buffer 244 inputs and stores the 
transformed data. A histogram generator 246 inputs the transformed image data from 
the data buffer 244 and generates a histogram on the image data. An estimated 

20 quantization table generator 248 inputs the image data from the histogram generator 
246 and generates one or more estimated quantization tables for the MxM blocks of 
image data. 

In operation, the data buffer 244 stores the minimum and maximum values of 

the compressed image blocks. The histogram generator 246 generates a histogram 
25 entry in a histogram for the block for each rounded block value. The quantization 

table generator 248 generates the quantization table by obtaining the maximum 

likelihood estimation of the quantization table. 

Fig. 4 is a flowchart outlining one embodiment of an image processing method 

in accordance with this invention. Beginning in step SI 000, control continues to step 
30 SI 100, where the decompressed image data is input. Next, in step SI 200, one or 

more estimated quantization tables are determined from the decompressed image data. 

Then, in step SI 400, the decompressed image is processed based, at least in part, on 
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the one or more estimated quantization tables. For example, the decompressed image 
data can be processed to remove compression artifacts based on the determined 
quantization table(s). It should be appreciated that, in step S1400, the image can be 
further processed based on any desired image processing technique. Control then 
5 continues to step SI 500. 

In step SI 500, the image is further processed. This further processing 
generally will not use the estimated quantization table(s). It should be appreciated 
that, even though Fig. 4 shows step SI 400 and step SI 500 as two separate steps, the 
image processing with and without the estimated quantization table(s) may be 

10 combined into one single step. Control then continues to step SI 600, where the 
control routine ends. 

Fig. 5 outlines in greater detail one exemplary embodiment of a method for 
determining the quantization table of step SI 200. Beginning in step SI 200, control 
continues to step S1210, where the next block of compressed image data is input as 

15 the current block. Next, in step SI 220, the minimum value and maximum value of 
the current block are determined. Control then continues to step S1230. 

In step S 1230, a determination is made whether the minimum value is equal to 
0, the maximum value is equal to 255, and/or the minimum and the maximum value 
are equal to each other. If so, the current block may contain truncated pixels or the 

20 block is uniform. Accordingly, this current block should not be analyzed. Control 

thus returns to step S1210. Otherwise, the current block is not excluded from analysis. 
Control therefore continues to step SI 240. 

In other words, if the maximum value is 255 or the minimum value is 0, the 
block may contain truncated values. If the maximum value is equal to the minimum 

25 value, the block is uniform. Both truncated and uniform blocks should be excluded 
from further processing. 

In step SI 240, a Discrete Cosine Transform is performed on each block. Next, 
in step S1250, each of the DCT values is rounded to an integer as {Y'(m,n)}. Control 
then continues to step SI 260. 

30 In step S 1260, a determination is made whether the current block is the last 

block in the image. If so, control continues to step SI 270. Otherwise, control returns 
to stepS1210 : 
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In step SI 270, the current DCT entry (m,n) is initialized as (0,0). Then, in 
step SI 280, a histogram for the absolute value of the rounded DCT value 
{ | Y*(m,n) | }is built. An exemplary embodiment of the implementation of a 
histogram is disclosed in copending application (Attorney Docket No. D/98495) 
incorporated herein by reference in its entirety. Next, in step SI 290, the highest count 
Q in the histogram is determined. As discussed above, the search is restricted to the 
highest peak and the two adjacent levels. Then, in step S1295, a determination is 
made whether the highest count is greater than zero. If the highest count is zero, the 
quantization value is unknown. Thus, control jumps to step S1380. Else, control 
continues to step SI 300. 

In step S1300, the maximum likelihood estimation of the quantization value 
q(m,n) is determined. Then in step SI 380, a determination is made whether (m,n) is 
the last entry in quantization table. If not, control continues to step S1385. 
Otherwise, control jumps to step SI 390 where control returns to step S1400. 

In step S1385, the current entry is set to the next entry, and the control returns 
to step SI 280. 

Fig. 6 outlines in greater detail one exemplary embodiment of a method for 
determining the maximum likelihood of step SI 300. 

Beginning in step SI 300, control continues to step SI 305, where the 
maximum likelihood is initialized to negative infinity. Next, in step S1310, the next 
level of the Q-l, Q and Q+l levels is input as the current level Ql, i.e. the current 
level Ql is one of the three levels Q-l, Q and Q+l, where Q is the level having the 
highest peak outside the main lobe. As discussed above, the search is restricted to 
only the Q-l, Q and Q+l levels. Control then continues to step S1315. 

In step S1315, the next value x of an index counter is input, where x is 
between 1 and Ql. That is, the estimation is performed up to the current level Ql. 
Then, in step SI 320, the initial estimation q is initialized as Ql/x. Next, in step 
SI 325, a determination is made whether q is an integer. If so, control continues to 
step SI 330. Else, control returns to S1315. 

In step SI 330, the likelihood is determined. Then, in step SI 340, a 
determination is made whether the determined likelihood is greater than the current 
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maximum likelihood. If not, then the determined likelihood is not the maximum 
likelihood. Control thus returns to step S1315. Else, control continues to step S1345. 

In step SI 345, the estimation value is set as the estimation q. Then, in step 
S1350, a determination is made whether x is greater than or equal to Ql. If not, 
5 control returns to step S1315. Else, the peak Ql is reached and control continues to 
step SI 355. 

In step SI 355, a determination is made whether all three levels have been 
searched. If not, control continues to step SI 360. Else, control jumps to step SI 365. 
Then, in step SI 360, Ql is incremented to the next level. Control then returns to step 
10 S1315. 

In contrast, in step SI 365, the maximum likelihood value is set to the 
estimation value. Control then continues to step SI 370, where control returns to step 
S1380. 

Fig. 7 outlines in greater detail one exemplary embodiment of the likelihood 
15 determination step SI 330 when q is an integer. 

Beginning in step SI 330, control continues to step SI 331, where the next 
counter z is set to q. Next, in step SI 332, z is decremented. That is, the estimation of 
the parameter is performed with values less than q, i.e., q-1 down to 0. Control then 
continues to step SI 333. 
20 In step SI 333, the number of blocks N(z) is calculated from the histogram h as 

Zj h(z + j q). Then, in step 1334, M is determined as min {q— 1, round[D(m)D(n)]}. 
Next, in step SI 335, a determination is made whether z> M and q-z >M and N(z) > 0. 
If so, control returns to step SI 331. Else, control continues to step SI 336. 

In step S1336, the likelihood is calculated as Zj N(i) log P d (i) + N log a*. 
25 Then in step SI 337, a determination is made whether z is equal to zero, i.e., that all 
values less than q have been analyzed. If not, control returns to step SI 332. 
Otherwise, control continues to step SI 338 where control returns to step SI 340. 

As shown in Fig. 1 , the image processing system 200 is preferably 
implemented on a programmed general purpose computer. However, the image 
30 processing system 200 can also be implemented on a special purpose computer, a 
programmed microprocessor or microcontroller and peripheral integrated circuit 
elements, an ASIC or other integrated circuit, a digital signal processor, a hardwired 




14 



electronic or logic circuit such as a discrete element circuit, a programmable logic 
device such as a PLD, PLA, FPGA or PAL, or the like. In general, any device, 
capable of implementing steps S1000-S1600 of Figs. 4-7 can be used to implement 
the image processing system. 
5 This invention has been described in connection with the preferred 

embodiments. However it should be understood that there is no intent to limit the 
invention to the embodiments described above. On the contrary, the intent to cover all 
alternatives, modification, and equivalents as may be included within the spirit and 
scope of the invention. 



